21 research outputs found
Equivalence of Systematic Linear Data Structures and Matrix Rigidity
Recently, Dvir, Golovnev, and Weinstein have shown that sufficiently strong
lower bounds for linear data structures would imply new bounds for rigid
matrices. However, their result utilizes an algorithm that requires an
oracle, and hence, the rigid matrices are not explicit. In this work, we derive
an equivalence between rigidity and the systematic linear model of data
structures. For the -dimensional inner product problem with queries, we
prove that lower bounds on the query time imply rigidity lower bounds for the
query set itself. In particular, an explicit lower bound of
for redundant storage bits would
yield better rigidity parameters than the best bounds due to Alon, Panigrahy,
and Yekhanin. We also prove a converse result, showing that rigid matrices
directly correspond to hard query sets for the systematic linear model. As an
application, we prove that the set of vectors obtained from rank one binary
matrices is rigid with parameters matching the known results for explicit sets.
This implies that the vector-matrix-vector problem requires query time
for redundancy in the systematic linear
model, improving a result of Chakraborty, Kamma, and Larsen. Finally, we prove
a cell probe lower bound for the vector-matrix-vector problem in the high error
regime, improving a result of Chattopadhyay, Kouck\'{y}, Loff, and
Mukhopadhyay.Comment: 23 pages, 1 tabl
Shattered Sets and the Hilbert Function
We study complexity measures on subsets of the boolean hypercube and exhibit connections between algebra (the Hilbert function) and combinatorics (VC theory). These connections yield results in both directions. Our main complexity-theoretic result demonstrates that a large and natural family of linear program feasibility problems cannot be computed by polynomial-sized constant-depth circuits. Moreover, our result applies to a stronger regime in which the hyperplanes are fixed and only the directions of the inequalities are given as input to the circuit. We derive this result by proving that a rich class of extremal functions in VC theory cannot be approximated by low-degree polynomials. We also present applications of algebra to combinatorics. We provide a new algebraic proof of the Sandwich Theorem, which is a generalization of the well-known Sauer-Perles-Shelah Lemma.
Finally, we prove a structural result about downward-closed sets, related to the Chvatal conjecture in extremal combinatorics
Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems
We consider the general problem of learning about a matrix through vector-matrix-vector queries. These queries provide the value of u^{T}Mv over a fixed field ? for a specified pair of vectors u,v ? ??. To motivate these queries, we observe that they generalize many previously studied models, such as independent set queries, cut queries, and standard graph queries. They also specialize the recently studied matrix-vector query model. Our work is exploratory and broad, and we provide new upper and lower bounds for a wide variety of problems, spanning linear algebra, statistics, and graphs. Many of our results are nearly tight, and we use diverse techniques from linear algebra, randomized algorithms, and communication complexity
Close Category Generalization for Out-of-Distribution Classification
Out-of-distribution generalization is a core challenge in machine learning.
We introduce and propose a solution to a new type of out-of-distribution
evaluation, which we call close category generalization. This task specifies
how a classifier should extrapolate to unseen classes by considering a
bi-criteria objective: (i) on in-distribution examples, output the correct
label, and (ii) on out-of-distribution examples, output the label of the
nearest neighbor in the training set. In addition to formalizing this problem,
we present a new training algorithm to improve the close category
generalization of neural networks. We compare to many baselines, including
robust algorithms and out-of-distribution detection methods, and we show that
our method has better or comparable close category generalization. Then, we
investigate a related representation learning task, and we find that performing
well on close category generalization correlates with learning a good
representation of an unseen class and with finding a good initialization for
few-shot learning. The code is available at
https://github.com/yangarbiter/close-category-generalizatio